1. Field of the Invention
The present invention is related to an image processing apparatus for enlarge-processing an image represented in a multi-gradation manner, and also is related to a program for executing the image enlarge-processing operation by utilizing a computer.
2. Description of the Related Art
As one of signal processing operations with respect to digital images, there are enlarging process operations. For instance, various sorts of image processing functions having high image qualities have been required in database fields and high-precision color printing field. As one of these image processing functions, image enlarging functions are required. This image enlarging process operation may constitute one of basic processing operations with respect to systems capable of editing, filing, displaying, and printing images. For instance, in such a case that digital images entered from external sources are printed out by printers having different resolution, or are displayed on display units in an enlarging manner, enlarging process operations of these digital images are required. For example, such an image enlarging operation is required in the case that image data standardized by 720 (horizontal direction)×480 (vertical direction) pixels is displayed on such a display unit having 800×600 pixels in a full screen mode. Also, this enlarging process operation may constitute a very important function which may also be used as a resolution converting method required so as to couple media having different resolution with each other, while the media are known as, for instance, the HDTV (High Definition Television) system, NTSC type television, electronic still cameras, medical imaging systems, and printing image systems. Furthermore, very recently, needs for enlarging process operations having high image qualities are positively made in such a case that since image data having relatively low resolution (display resolution) of approximately 75 [dpi] are mainly displayed are popularized, these low resolution image data (e.g., images of home pages on the Internet and digital video) are printed out by high-resolution printers, so as to obtain print-out results having high image qualities.
As methods for enlarge-processing multi-value images represented in a multi-gradation mode, namely methods for acquiring values of respective pixel positions after being enlarged, conventionally, a large number of methods have been proposed, for instance, such enlarging systems for basically employing interpolations (will also be referred to as “interpolation enlarging systems” hereinafter) have been well known, e.g., the nearest neighbor method, the bilinear method, and the cubic convolution method.
The nearest neighbor method corresponds to such a method that when pixels are inverse-mapped onto an original image, a pixel value of such a pixel located at the nearest distance thereof is used as each of pixel values after being enlarged. For example, assuming now that an enlarging ratio along an x direction is ‘a’, and an enlarging ratio along a y direction is “b”, a calculation is made of inverse-mapping points on the original image, where the respective coordinate points (X, Y) after being enlarged are enlarged by 1/a and 1/b, respectively, and then, such a pixel value on the original image, which are located at the nearest distance, are set as the pixel values (X, Y).
The bilinear method corresponds to the following method. That is, assuming now that pixel values among pixels are changed in a linear manner, a region surrounded by pixel points (4 points in four neighbor pixels) along the x direction and the y direction is linear-approximated (linear-interpolated) with reference to pixels (e.g., 4 neighbor pixels) located in the vicinity of a point where a pixel after being enlarged is inverse-mapped so as to acquire a pixel value at the inverse-mapped point. In this bilinear method, although the processing load thereof becomes larger than that of the nearest neighbor method, the calculation amount thereof is relatively smaller than that of this nearest neighbor method. Also, since the linear interpolation itself owns the smoothing effect, jaggy can hardly appear, as compared with the nearest neighbor method.
The cubic convolution method corresponds to such a method. That is, while an interpolation function for approximating a sinc function {sinc(x)=sin(x)/x} is defined based upon the sampling theory, neighbor pixels as to a point where pixels after being enlarged are inverse-mapped are convoluted with the above-described approximate interpolation function so as to acquire pixel values after being enlarged. These neighbor pixels are, for instance, 16 pixels constructed of 4 pixels along x direction and Y direction.
This cubic convolution method is established based upon the following idea. That is, while utilizing such an ideal characteristic that the frequency characteristic of the sinc function is equal to “1” within the Nyquist frequency and is equal to “0” outside this Nyqust frequency, folding distortions caused by resembling operation are suppressed. Since this cubic convolution method can produce a sharp image, the resultant image quality thereof becomes relatively better than that of the above-described two other methods.
However, these interpolating/enlarging methods own such a trend that a blurring phenomenon essentially occurs. Although the nearest neighbor method can execute the high-speed processing operation due to the simple processing operation and the small calculation amount, since one pixel of an original image is directly enlarged in a rectangular shape, there are large degrees of visible image quality deteriorations. For instance, in the case that either an inclined line or a boundary line is present in an original image, a zig-zag-shaped deterioration called as “jaggy” may be produced in an edge portion and an inclined line portion contained in an enlarged image, and/or when magnification is large, and enlarged image becomes a mosaic-shape (block-image-shaped). In the bilinear method, since the smoothing (low-pass filtering) effect is emphasized, an entire image may become blurred. That is, edge portions are mainly smoothed by the low-pass filtering effect, while such an assumption that an image is changed in a linear manner cannot be applied to the edge portions. Since the calculation amount of the cubic convolution method becomes larger than the calculation amounts of other two methods and the reference range thereof become larger, this cubic convolution method is not suitable when high-speed processing operation is required. Also, since the sinc function corresponds to an infinitely continued function, this cubic convolution method owns a high-frequency-range-emphasized characteristic which is caused by that this infinitely continued function is cut within a predetermined range (namely, −2 to +2 in an example of 16 pixels). As a result, a small amount of so-called “jaggy” may be produced in an edge portion, and a noise component may be emphasized, which are not so stronger than those of the nearest neighbor method.
To the contrary, very recently, as such an approach which is completely different from the above-described interpolating/enlarging methods, an enlarging method capable of preventing occurrences of blurring phenomenon, jaggy, and block distortions has been proposed by utilizing iterated conversion coding operations. For instance, there are certain enlarging methods using the fractal conceptional idea among these iterated conversion coding operations (for instance, U.S. Pat. No. 5,065,447 etc.). Also, the normal IFS (Iterated Function System) is employed so as to utilize the fractal conceptional idea. An enlarging system established based upon the iterated conversion coding operation by using the fractal conceptional idea will now be referred to as a “fractal enlarging manner.”
The fractal conceptional idea implies that a self-similar characteristic of an image under such an initial condition that when a portion of an image is derived from an entire image, another image which is better resembled to this derived image is present within this image, while having a different size. Then, the fractal enlarging method owns such a merit that since a block image distortion does not appear and furthermore self-similar characteristic established between block images having different sizes is utilized within an image, the fractal enlarged image does not depend upon resolution during decoding operation. Also, this fractal enlarging method can obtain an enlarged image having a high image quality even in a relatively large enlarging ratio. For example, the above-described U.S. Pat. No. 5,065,447 has proposed the method for acquiring the enlarged image as follows. That is, as to an initial image having an enlarged size, such a range block image which has been enlarged in the same enlarging ratio due to the enlarged initial image is coordinate-converted and pixel-value-converted. Also, such a process operation is repeated carried out for replacing the converted position of the range block image by the position of the domain block image enlarged in the same enlarging ratio. In this fractal enlarging method, while the feature of the fractal conceptional idea is utilized, the enlarged image having a less blurring component can be obtained by suppressing the occurrence of jaggy.
However, in the enlarging method using the iterated conversion coding operation, since the enlarged image is produced by iterating a predetermined process operation, the processing time thereof would become considerably longer than that of the interpolating/enlarging method. For instance, in the fractal enlarging method, the processing time is increased by seeking the range block images. Furthermore, the visibly-allowable enlarged image can be hardly produced by merely executing the substituting process operation into the domain block image only one time. As a result, since these process operations are iterated so as to acquire the enlarged image, the resultant processing time required for obtaining the visibly-allowable enlarged image would be prolonged. Also, this fractal enlarging method newly owns another image quality deterioration problem. That is, as to a document image and a stepped edge portion, block distortions may occur, noise-shaped trash (smear) may he produced, and oozing of splinter-shaped pixel values may occur. Also, the conventional fractal enlarging method has a problem such that reproducibility of a busy portion is deteriorated (namely, blurring phenomenon and noise).
As the trial methods capable of solving the problems related to the image qualities owned by the fractal enlarging method, for instance, some methods have been proposed in U.S. Pat. No. 6,141,017, Japanese Laid-open Patent Application No. HEI 11-331595, and Japanese Laid-open Patent Application No. HEI 11-8758.
The above-described U.S. Pat. No. 6,141,017 has proposed the method for suppressing the discontinuity characteristic occurred in the boundary between the block images in such a way that the domain block images are produced in the overlapping manner, and when the range block image is substituted, only the inside portion of the range block image is substituted for the inside portion corresponding to the domain block image. Also, this US patent has proposed the method for performing the enlarging process operation capable of reproducing the edge portion under better condition by way of the fractal enlarging method in such a manner that the block image is divided into the edge portion and the flat portion from dispersion of the pixel values contained in the block image which is required when the pixel value converting coefficient is calculated, while suppressing that the flat portion becomes the pictorial tone by employing another method with respect to the flat portion. However, this conventional method still cannot solve the deterioration problem such that the noise-shaped trash occurs at the stepped edge portion, and also, the splinter-shaped pixel value oozes.
Also, the above-explained Japanese Laid-open Patent Application No. HEI-11-331595 has proposed such a method capable of suppressing the discontinuity characteristic at the boundary between the block images in such a manner. That is, while both the fractal enlarging method and the bilinear enlarging method are applied to the image, when a difference thereof is small, the process result obtained by the fractal enlarging method is employed, whereas when a difference thereof is large, the result obtained by the bilinear method is employed. Alternatively, the results are blended to each other in response to the magnitude of the difference. However, this conventional method owns such a trend that the bilinear method is selected at the edge portion where the difference between the bilinear enlarging method and the fractal enlarging method. As a consequence, this conventional method owns such a drawback that the clear edge reproduction (namely, feature achieved by fractal enlarging method) cannot be realized, so that the resultant image becomes blurred.
Also, the above-described Japanese Laid-open Patent Application No. HEI-11-8758 has proposed such a method capable of performing a seeking operation in a high speed by that while the histogram means for storing the use frequency of the range block images is provided, such a range block image whose use frequency is low is not used in the seeking operation. However, since the restriction in the range block image seeking range based upon the use frequency has no reason in view of the image quality in this method, there is such a drawback that this restriction may cause the image quality to be deteriorated.
As previously explained, the conventional methods capable of improving the fractal enlarging method have separately proposed the improving ideas with respect to the image quality problems and the processing time problems. These image quality problems are known as the jaggy, the blurring phenomenon, and the block distortions, which may occur when the enlarged images are produced. However, these problems could not yet been solved in a comprehensive manner.